Information processing method, information processing device, and program

ABSTRACT

An information processing method for causing a computer to process an image, wherein the image processing method causes the computer to execute an acquisition step of acquiring the image, a first production step of producing an editing image for editing at least a portion of the image and a first changing image for changing the editing image to be output, an editing step of editing the image on the produced editing image, a second production step of producing a second changing image based on an image edited at the editing step, and an output step of outputting an output image that has at least the editing image and the second changing image.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation application filed under 35 U.S.C.111(a) claiming the benefit under 35 U.S.C. 120 of U.S. patentapplication Ser. No. 15/671,338 filed on Aug. 8, 2017, which is acontinuation application filed under 35 U.S.C. 111(a) claiming thebenefit under 35 U.S.C. 120 of U.S. patent application Ser. No.14/924,871 filed on Oct. 28, 2015, which is a continuation applicationfiled under 35 U.S.C. 111(a) claiming the benefit under 35 U.S.C. 120and 365(c) of PCT International Application No.PCT/JP2015/057609 filedon Mar. 10, 2015, which is based upon and claims priority to JapanesePatent Application No. 2014-054782 filed on Mar. 18, 2014, the entirecontents of which are incorporated herein by reference.

BACKGROUND OF THE INVENTION 1. Field of the Invention

The present invention relates to at least one of an informationprocessing method, an information processing device, and a program.

2. Description of the Related Art

A method for displaying a panoramic image has been known conventionally.

A user interface (that will be referred to as a “UI” below) has beenknown for accepting an instruction from a user with respect to displayof a panoramic image in a panoramic image display (see, for example,Japanese Patent Application Publication No. 2011-076249).

However, a conventional UI assigns a function for scrolling an image toso-called “dragging” at a time of image display on a smartphone or thelike, and hence, it may be difficult for a user to execute an imageoperation, for example, editing of an image or the like.

SUMMARY OF THE INVENTION

According to one aspect of the present invention, there is provided aninformation processing method for causing a computer to process animage, wherein the image processing method causes the computer toexecute an acquisition step of acquiring the image, a first productionstep of producing an editing image for editing at least a portion of theimage and a first changing image for changing the editing image to beoutput, an editing step of editing the image on the produced editingimage, a second production step of producing a second changing imagebased on an image edited at the editing step, and an output step ofoutputting an output image that has at least the editing image edited atthe editing step and the second changing image produced at the secondproduction step.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram that illustrates one example of an entireconfiguration of an image taking system according to one embodiment ofthe present invention.

FIG. 2A, FIG. 2B, and FIG. 2C are diagrams that illustrate one exampleof an image taking device according to one embodiment of the presentinvention.

FIG. 3 is a diagram that illustrates one example of image taking by animage taking device according to one embodiment of the presentinvention.

FIG. 4A, FIG. 4B, and FIG. 4C are diagrams that illustrate one exampleof an image taken by an image taking device according to one embodimentof the present invention.

FIG. 5 is a block diagram that illustrates one example of a hardwareconfiguration of an image taking device according to one embodiment ofthe present invention.

FIG. 6 is a block diagram that illustrates one example of a hardwareconfiguration of a smartphone according to one embodiment of the presentinvention.

FIG. 7 is a sequence diagram that illustrates one example of an entireprocess of an image taking system according to one embodiment of thepresent invention.

FIG. 8A, FIG. 8B, FIG. 8C, and FIG. 8D are diagrams that illustrate oneexample of an all celestial sphere image according to one embodiment ofthe present invention.

FIG. 9 is a diagram that illustrates one example of an all celestialsphere panoramic image according to one embodiment of the presentinvention.

FIG. 10A, FIG. 10B, FIG. 100, and FIG. 10D are diagrams for illustratingone example of an initial image according to one embodiment of thepresent invention.

FIG. 11 is a diagram that illustrates one example of an output image atan initial state for executing editing of an image according to oneembodiment of the present invention.

FIG. 12A, FIG. 12B, and FIG. 12C are diagrams for illustrating oneexample of editing of an area to be output according to one embodimentof the present invention.

FIG. 13A and FIG. 13B are diagrams for illustrating one example ofenlargement or reduction of an area to be output according to oneembodiment of the present invention.

FIG. 14 is a diagram for illustrating one example of another zoomprocess according to one embodiment of the present invention.

FIG. 15 is a table for illustrating one example of another zoom processaccording to one embodiment of the present invention.

FIG. 16A, FIG. 16B, FIG. 16C, FIG. 16D, and FIG. 16E are diagrams forillustrating one example of a “range” of another zoom process accordingto one embodiment of the present invention.

FIG. 17A and FIG. 17B are diagrams for illustrating one example ofediting that is executed for a predetermined area based on an editingimage according to one embodiment of the present invention.

FIG. 18 is a flowchart that illustrates one example of an entire processof a smartphone according to one embodiment of the present invention.

FIG. 19A and FIG. 19B are diagrams for illustrating one example ofchanging of an output such as a position or direction of a changingimage according to one embodiment of the present invention.

FIG. 20 is a functional diagram for illustrating one example of afunctional configuration of an image taking system according to oneembodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

An embodiment of the present invention will be described below.

First Embodiment <An Entire Configuration of a System>

FIG. 1 is a diagram that illustrates one example of an entireconfiguration of an image taking system according to one embodiment ofthe present invention.

An image taking system 10 has an image taking device 1 and a smartphone2.

The image taking device 1 has a plurality of optical systems, andproduces, and outputs to the smartphone 2, for example, a taken image ofa wide range such as all directions around the image taking device 1(that will be referred to as an “all celestial sphere image” below).Details of the image taking device 1 and an all celestial sphere imagewill be described below. An image that is processed by the image takingsystem 10 is, for example, an all celestial sphere image. A panoramicimage is, for example, an all celestial sphere image. An example of anall celestial sphere image will be described below.

An information processing device is, for example, the smartphone 2. Thesmartphone 2 will be described as an example below. The smartphone 2 isa device for causing a user to operate an all celestial sphere imageacquired from the image taking device 1. The smartphone 2 is a devicefor causing a user to output an acquired all celestial sphere image. Adetail of the smartphone 2 will be described below.

The image taking device 1 and the smartphone 2 are subjected to wired orwireless connection. For example, the smartphone 2 downloads from theimage taking device 1, and inputs to the smartphone 2, data such an allcelestial sphere image output from the image taking device 1. Here,connection may be executed through a network.

Here, an entire configuration is not limited to a configurationillustrated in FIG. 1. For example, the image taking device 1 and thesmartphone 2 may be an integrated device. Furthermore, another computerother than the image taking device 1 and the smartphone 2 may beconnected to be composed of three or more devices.

<An Image Taking Device>

FIG. 2A, FIG. 2B, and FIG. 2C are diagrams that illustrate one exampleof an image taking device according to one embodiment of the presentinvention.

FIG. 2A, FIG. 2B, and FIG. 2C are diagrams that illustrate one exampleof an appearance of the image taking device 1. FIG. 2A is one example ofan elevation view of the image taking device 1. FIG. 2B is one exampleof a left side view of the image taking device 1. FIG. 2C is one exampleof a plan view of the image taking device 1.

The image taking device 1 has a front side image taking element 1H1, aback side image taking element 1H2, and a switch 1H3. A hardware that isprovided in an interior of the image taking device 1 will be describedbelow.

The image taking device 1 produces an all celestial sphere image byusing images taken by the front side image taking element 1H1 and theback side image taking element 1H2.

The switch 1H3 is a so-called “shutter button” and is an input devicefor causing a user to execute an instruction of image taking for theimage taking device 1.

The image taking device 1 is held by hand of a user, for example, asillustrated in FIG. 2A, and the switch 1H3 is pushed to execute imagetaking.

FIG. 3 is a diagram that illustrates one example of image taking by animage taking device according to one embodiment of the presentinvention.

As illustrated in FIG. 3, a user holds the image taking device 1 by handand pushes the switch 1H3 in FIG. 2A, FIG. 2B, and FIG. 2C to executeimage taking. As illustrated in FIG. 3, it is possible for the imagetaking device 1 to take an image in all directions around the imagetaking device 1 by the front side image taking element 1H1 in FIG. 2A,FIG. 2B, and FIG. 2C and the back side image taking element 1H2 in FIG.2A, FIG. 2B, and FIG. 2C.

FIG. 4A, FIG. 4B, and FIG. 4C are diagrams that illustrate one exampleof an image taken by an image taking device according to one embodimentof the present invention.

FIG. 4A is one example of an image taken by the front side image takingelement 1H1 in FIG. 2A, FIG. 2B, and FIG. 2C. FIG. 4B is one example ofan image taken by the back side image taking element 1H2 in FIG. 2A,FIG. 2B, and FIG. 2C. FIG. 4C is one example of an image that isproduced based on an image taken by the front side image taking element1H1 in FIG. 2A, FIG. 2B, and FIG. 2C and an image taken by the back sideimage taking element 1H2 in FIG. 2A, FIG. 2B, and FIG. 2C.

An image taken by the front side image taking element 1H1 in FIG. 2A,FIG. 2B, and FIG. 2C is an image with an image taking range that is awide range in a front direction of the image taking device 1, forexample, a range of 180° as an angle of view, as illustrated in FIG. 4A.An image taken by the front side image taking element 1H1 in FIG. 2A,FIG. 2B, and FIG. 2C has a distortion aberration as illustrated in FIG.4A, in a case where the front side image taking element 1H1 in FIG. 2A,FIG. 2B, and FIG. 2C uses an optical system for taking an image with awide range, for example, a so-called “fisheye lens”. An image in FIG. 4Ataken by the front side image taking element 1H1 in FIG. 2A, FIG. 2B,and FIG. 2C is a so-called “hemispherical image” that has a wide rangein one side of the image taking device 1 and a distortion aberration(that will be referred to as a “hemispherical image” below).

Here, it is desirable for an angle of view to be within a range greaterthan or equal to 180° and less than or equal to 200°. In particular, asa hemispherical image in FIG. 4A and a hemispherical image in FIG. 4Bthat will be described below are synthesized in a case where an angle ofview is greater than 180°, there is an overlapping image area, andhence, synthesis is facilitated.

An image taken by the back side image taking element 1H2 in FIG. 2A,FIG. 2B, and FIG. 2C is an image with an image taking range that is awide range in a back direction of the image taking device 1, forexample, a range of 180° as an angle of view, as illustrated in FIG. 4B.

An image in FIG. 4B taken by the back side image taking element 1H2 inFIG. 2A, FIG. 2B, and FIG. 2C is a hemispherical image similar to thatof FIG. 4A.

The image taking device 1 executes processes such as a distortioncorrection process and a synthesis process, and thereby produces animage illustrated in FIG. 4C from a front side hemispherical image inFIG. 4A and a back side hemispherical image in FIG. 4B. FIG. 4C is animage produced by, for example, Mercator's projection, equidistantcylindrical projection, or the like, namely, an all celestial sphereimage.

Here, an all celestial sphere image is not limited to an image producedby the image taking device 1. An all celestial sphere image may be, forexample, an image taken by another camera or the like, or an imageproduced based on an image taken by another camera. It is desirable foran all celestial sphere image to be an image with a view angle in a widerange taken by a so-called “all direction camera”, a so-called “wideangle lens camera”, or the like.

Furthermore, an all celestial sphere image will be described as anexample, and an image is not limited to such an all celestial sphereimage. An image may be, for example, an image or the like taken by acompact camera, a single lens reflex camera, a smartphone, or the like.An image may be a panoramic image that extends horizontally orvertically, or the like.

<A Hardware Configuration of an Image Taking Device>

FIG. 5 is a block diagram that illustrates one example of a hardwareconfiguration of an image taking device according to one embodiment ofthe present invention.

The image taking device 1 has an image taking unit 1H4, an imageprocessing unit 1H7, an image control unit 1H8, a Central ProcessingUnit (CPU) 1H9, and a Read-Only Memory (ROM) 1H10. Furthermore, theimage taking device 1 has a Static Random Access Memory (SRAM) 1H11, aDynamic Random Access Memory (DRAM) 1H12, and an operation interface(I/F) 1H13. Moreover, the image taking device 1 has a network I/F 1H14,a wireless I/F 1H15, and an antenna 1H16. Each component of the imagetaking device 1 is connected through a bus 1H17 and executes input oroutput of data or a signal.

The image taking unit 1H4 has the front side image taking element 1H1and the back side image taking element 1H2. A lens 1H5 that correspondsto the front side image taking element 1H1 and a lens 1H6 thatcorresponds to the back side image taking element 1H2 are placed. Thefront side image taking element 1H1 and the back side image takingelement 1H2 are so-called “camera units”. The front side image takingelement 1H1 and the back side image taking element 1H2 have opticalsensors such as a Complementary Metal Oxide semiconductor (CMOS) or aCharge Coupled Device (CCD). The front side image taking element 1H1executes a process for converting light incident on the lens 1H5 toproduce image data. The back side image taking element 1H2 executes aprocess for converting light incident on the lens 1H6 to produce imagedata. The image taking unit 1H4 outputs image data produced by the frontside image taking element 1H1 and the back side image taking element 1H2to the image processing unit 1H7. For example, image data are the frontside hemispherical image in FIG. 4A and the back side hemisphericalimage in FIG. 4B or the like.

Here, the front side image taking element 1H1 and the back side imagetaking element 1H2 may have an optical element other than a lens, suchas a stop or a low-pass filter, in order to execute image taking with ahigh image quality. Furthermore, the front side image taking element 1H1and the back side image taking element 1H2 may execute a process such asa so-called “defective pixel correction” or a so-called “hand movementcorrection” in order to execute image taking with a high image quality.

The image processing unit 1H7 executes a process for producing an allcelestial sphere image in FIG. 4C from image data that are input fromthe image taking unit 1H4. A detail of a process for producing an allcelestial sphere image will be described below. Here, a process that isexecuted by the image processing unit 1H7 may be such that a part or anentirety of a process is executed in parallel and redundantly by anothercomputer.

The image taking control unit 1H8 is a control device that controls eachcomponent of the image taking device 1.

The CPU 1H9 executes an operation or a control for each process that isexecuted by the image taking device 1. For example, the CPU 1H9 executeseach kind of program. Here, the CPU 1H9 may be composed of a pluralityof CPUs or devices or a plurality of cores in order to attainspeeding-up due to parallel processing. Furthermore, a process of theCPU 1H9 may be such that another hardware resource is provided inside oroutside the image taking device 1 and caused to execute a part or anentirety of a process for the image taking device 1.

The ROM 1H10, the SRAM 1H11, and the DRAM 1H12 are examples of a storagedevice. The ROM 1H10 stores, for example, a program, data, or aparameter that is executed by the CPU 1H9. The SRAM 1H11 and the DRAM1H12 store, for example, a program, data to be used in a program, datato be produced by a program, a parameter, or the like, in a case wherethe CPU 1H9 executes a program. Here, the image taking device 1 may havean auxiliary storage device such as a hard disk.

The operation I/F 1H13 is an interface that executes a process forinputting an operation of a user to the image taking device 1, such asthe switch 1H3. The operation I/F 1H13 is an operation device such as aswitch, a connector or cable for connecting an operation device, acircuit for processing a signal input from an operation device, adriver, a control device, or the like. Here, the operation I/F 1H13 mayhave an output device such as a display. Furthermore, the operation I/F1H13 may be a so-called “touch panel” wherein an input device and anoutput device are integrated, or the like. Moreover, the operation I/F1H13 may have an interface such as a Universal Serial Bus (USB), connecta storage medium such as Flash Memory (“Flash Memory” is a registeredtrademark), and input from and output to the image taking device 1,data.

Here, the switch 1H3 may have an electric power source switch forexecuting an operation other than a shutter operation, a parameter inputswitch, or the like.

The network I/F 1H14, the wireless I/F 1H15, and the antenna 1H16 aredevices for connecting the image taking device 1 with another computerthrough a wireless or wired network and a peripheral circuit or thelike. For example, the image taking device 1 is connected to a networkthrough the network I/F 1H14 and transmits data to the smartphone 2.Here, the network I/F 1H14, the wireless I/F 1H15, and the antenna 1H16may be configured to be connected by using a connector such as a USB, acable, or the like.

The bus 1H17 is used for an input or an output of data or the likebetween respective components of the image taking device 1. The bus 1H17is a so-called “internal bus”. The bus 1H17 is, for example, aPeripheral Component Interconnect Bus Express (PCI Express).

Here, the image taking device 1 is not limited to a case of two imagetaking elements. For example, it may have three or more image takingelements. Moreover, the image taking device 1 may change an image takingangle of one image taking element to take a plurality of partial images.Furthermore, the image taking device 1 is not limited to an opticalsystem that uses a fisheye lens. For example, a wide angle lens may beused.

Here, a process that is executed by the image taking device 1 is notlimited to that is executed by the image taking device 1. A part or anentirety of a process that is executed by the image taking device 1 maybe executed by the smartphone 2 or another computer connected through anetwork while the image taking device 1 may transmit data or aparameter.

<A Hardware Configuration of an Information Processing Device>

FIG. 6 is a block diagram that illustrates one example of a hardwareconfiguration of an information processing device that includes asmartphone according to one embodiment of the present invention.

An information processing device is a computer. An informationprocessing device may be, for example, a notebook Personal Computer(PC), a Personal Digital Assistance (PDA), a tablet, a mobile phone, orthe like, other than a smartphone.

The smartphone 2 that is one example of an information processing devicehas an auxiliary storage device 2H1, a main storage device 2H2, aninput/output device 2H3, a state sensor 2H4, a CPU 2H5, and a networkI/F 2H6. Each component of the smartphone 2 is connected to a bus 2H7and executes an input or an output of data or a signal.

The auxiliary storage device 2H1 stores information such as each kind ofdata that includes an intermediate result of a process executed by theCPU 2H5 due to a control of the CPU 2H5, a control device, or the like,a parameter, or a program. The auxiliary storage device 2H1 is, forexample, a hard disk, a flash Solid State Drive (SSD), or the like.Here, information stored in the auxiliary storage device 2H1 is suchthat a part or an entirety of such information may be stored in a fileserver connected to the network I/F 2H6 or the like, instead of theauxiliary storage device 2H1.

The main storage device 2H2 is a main storage device such as a storagearea to be used by a program that is executed by the CPU 2H5, that is, aso-called “Memory”. The main storage device 2H2 stores information suchas data, a program, or a parameter. The main storage device 2H2 is, forexample, a Static Random Access Memory (SRAM), a DRAM, or the like. Themain storage device 2H2 may have a control device for executing storagein or acquisition from a memory.

The input/output device 2H3 is a device that has functions of an outputdevice for executing display and an input device for inputting anoperation of a user.

The input/output device 2H3 is a so-called “touch panel”, a “peripheralcircuit”, a “driver”, or the like.

The input/output device 2H3 executes a process for displaying, to auser, an image input in, for example, a predetermined Graphical UserInterface (GUI) or the smartphone 2.

The input/output device 2H3 executes a process for inputting anoperation of a user, for example, in a case where a GUI with a displayor an image is operated by such a user.

The state sensor 2H4 is a sensor for detecting a state of the smartphone2. The state sensor 2H4 is a gyro sensor, an angle sensor, or the like.The state sensor 2H4 determines, for example, whether or not one sidethat is possessed by the smartphone 2 is provided at a predetermined orgreater angle with respect to a horizon. That is, the state sensor 2H4executes a detection as to whether the smartphone 2 is provided at astate of a longitudinally directional attitude or a state of a laterallydirectional attitude.

The CPU 2H5 executes a calculation in each process that is executed bythe smartphone 2 and a control of a device that is provided in thesmartphone 2. For example, the CPU 2H5 executes each kind of program.Here, the CPU 2H5 may be composed of a plurality of CPUs or devices, ora plurality of cores in order to execute a process in parallel,redundantly, or dispersedly. Furthermore, a process for the CPU 2H5 issuch that another hardware resource may be provided inside or outsidethe smartphone 2 to execute a part or an entirety of a process for thesmartphone 2. For example, the smartphone 2 may have a GraphicsProcessing Unit (GPU) for executing image processing, or the like.

The network I/F 2H6 is a device such as an antenna, a peripheralcircuit, a driver, or the like, for inputting or outputting data, or thelike, that is connected to another computer through a wireless or wirednetwork. For example, the smartphone 2 executes a process for inputtingimage data from the image taking device 1 due to the CPU 2H5 and thenetwork I/F 2H6. The smartphone 2 executes a process for outputting apredetermined parameter or the like to the image taking device 1 due tothe CPU 2H5 and the network I/F 2H6.

<An Entire Process for an Image Taking System>

FIG. 7 is a sequence diagram that illustrates one example of an entireprocess for an image taking system according to one embodiment of thepresent invention.

At step S0701, the image taking device 1 executes a process forproducing an all celestial sphere image.

FIG. 8A, FIG. 8B, FIG. 8C, and FIG. 8D are diagrams that illustrate oneexample of an all celestial sphere image according to one embodiment ofthe present invention.

FIG. 8A, FIG. 8B, FIG. 8C, and FIG. 8D are diagrams that illustrate oneexample of a process for producing an all celestial sphere image at stepS0701.

FIG. 8A is a diagram illustrated in such a manner that positions in ahemispherical image in FIG. 4A where incidence angles are equal in ahorizontal direction or a vertical direction with respect to an opticalaxis are connected by a line. An incidence angle θ in a horizontaldirection with respect to an optical axis and an incidence angle ϕ in avertical direction with respect to such an optical axis will be denotedbelow.

Similarly to FIG. 8A, FIG. 8B is a diagram illustrated in such a mannerthat positions in a hemispherical image in FIG. 4B where incidenceangles are equal in a horizontal direction or a vertical direction withrespect to an optical axis are connected by a line.

FIG. 8C is a diagram that illustrates one example of an image processedin accordance with Mercator projection. FIG. 8C is an example of a casewhere an image in a state illustrated in FIG. 8A or FIG. 8B is, forexample, caused to correspond to a preliminarily produced Look Up Table(LUT) or the like and processed in accordance with equidistantcylindrical projection.

FIG. 8D is one example of a synthesis process for synthesizing imagesprovided by applying a process illustrated in FIG. 8C to FIG. 8A andFIG. 8B.

As illustrated in FIG. 8D, a synthesis process is to produce an image byusing a plurality of images, for example, in a state illustrated in FIG.8C. Here, a synthesis process is not limited to a process for simplyarranging pre-processed images successively. For example, in a casewhere a center of an all celestial sphere image in a horizontaldirection is not provided at θ=180°, a synthesis process may be aprocess for executing a synthesis process in such a manner that apre-processed image in FIG. 4A is arranged at a center of an allcelestial sphere image and a pre-processed image in FIG. 4B is dividedand arranged at left and right sides thereof, so as to produce an allcelestial sphere image illustrated in FIG. 4C.

Here, a process for producing an all celestial sphere image is notlimited to a process in accordance with equidistant cylindricalprojection. For example, a so-called “upside-down” case is provided insuch a manner that, like FIG. 8B, an alignment of pixels in a directionof ϕ is upside-down with respect to an alignment in FIG. 8A and analignment of pixels in a direction of θ is left-right reversal withrespect to an alignment in FIG. 8A. In an upside-down case, the imagetaking device 1 may execute a process for rolling or rotating apre-processed image in a state of FIG. 8B by 180° so as to align with analignment of pixels in a direction of ϕ and a direction of θ in FIG. 8A.

Furthermore, a process for producing an all celestial sphere image mayexecute a correction process for correcting distortion aberration thatis provided in an image in a state of FIG. 8A or FIG. 8B. Moreover, aprocess for producing an all celestial sphere image may execute aprocess for improving an image quality, for example, shading correction,gamma correction, white balance, hand movement correction, an opticalblack correction process, a defective pixel correction process, an edgeenhancement process, a linear correction process, or the like.

Here, for example, in a case where an image taking range of ahemispherical image overlaps with an image taking range of anotherhemispherical image, a synthesis process may execute correction byutilizing an overlapping range to execute such a synthesis process athigh precision.

Due to a process for producing an all celestial sphere image, the imagetaking device 1 produces an all celestial sphere image from ahemispherical image that is taken by the image taking device 1.

At step S0702, the smartphone 2 executes a process for acquiring an allcelestial sphere image produced at step S0701. A case where thesmartphone 2 acquires an all celestial sphere image in FIG. 8D will bedescribed as an example below.

At step S0703, the smartphone 2 produces an all celestial spherepanoramic image from an all celestial sphere image acquired at stepS0702.

FIG. 9 is a diagram that illustrates one example of an all celestialsphere panoramic image according to one embodiment of the presentinvention.

At step S0703, the smartphone 2 executes a process for producing an allcelestial sphere panoramic image in FIG. 9 from an all celestial sphereimage in FIG. 8D. An all celestial sphere panoramic image is an imageprovided in such a manner that an all celestial sphere image is appliedonto a spherical shape.

A process for producing an all celestial sphere panoramic image isrealized by, for example, an Application Programming Interface (API)such as Open GL (“Open GL” is a registered trademark) for EmbeddedSystems (Open GL ES).

An all celestial sphere panoramic image is produced by dividing an imageinto triangles, joining vertices P of triangles (that will be referredto as “vertices P” below), and applying a polygon thereof.

At step S0704, the smartphone 2 executes a process for causing a user toinput an operation for starting an output of an image. At step S0704,the smartphone 2, for example, reduces and outputs an all celestialsphere panoramic image produced at step S0703, that is, displays aso-called “thumbnail image”. In a case where a plurality of allcelestial sphere panoramic images are stored in the smartphone 2, thesmartphone 2 outputs a list of thumbnail images, for example, to cause auser to select an image to be output. At step S0704, the smartphone 2executes, for example, a process for inputting an operation for causinga user to select one image from a list of thumbnail images.

At step S0705, the smartphone 2 executes a process for producing aninitial image based on an all celestial sphere panoramic image selectedat step S0704.

FIG. 10A, FIG. 10B, FIG. 100, and FIG. 10D are diagrams for illustratingone example of an initial image according to one embodiment of thepresent invention.

FIG. 10A is a diagram that illustrates a three-dimensional coordinatesystem for illustrating one example of an initial image according to oneembodiment of the present invention.

As illustrated in FIG. 10A, a three-dimensional coordinate system withXYZ axes will be described below. The smartphone 2 places a virtualcamera 3 at a position of an origin and produces each kind of image at aviewpoint of the virtual camera 3. In a case of a coordinate system inFIG. 10A, an all celestial sphere panoramic image is represented by, forexample, a sphere CS. The virtual camera 3 corresponds to a viewpoint ofa user that views an all celestial sphere panoramic image wherein suchan all celestial sphere panoramic image is a sphere CS at a placedposition thereof.

FIG. 10B is a diagram for illustrating one example of a predeterminedarea for a virtual camera according to one embodiment of the presentinvention.

FIG. 10B is a case where FIG. 10A is represented by three-plane figures.FIG. 10B is a case where the virtual camera 3 is placed at an origin ofFIG. 10A. FIG. 10C is a projection view of one example of apredetermined area for a virtual camera according to one embodiment ofthe present invention.

A predetermined area T is an area where a view angle of the virtualcamera 3 is projected onto a sphere CS. The smartphone 2 produces animage based on a predetermined area T.

FIG. 10D is a diagram for illustrating one example of information fordetermining a predetermined area for a virtual camera according to oneembodiment of the present invention.

A predetermined area T is determined by, for example, predetermined areainformation (x, y, α).

A view angle α is an angle that indicates an angle of the virtual camera3 as illustrated in FIG. 10D. In a case of a diagonal angle of view 2Lof a predetermined area T that is represented by a view angle α,coordinates of a center point CP of such a predetermined area T arerepresented by (x,y) in predetermined area information.

Here, a distance from the virtual camera 3 to a center point CP isrepresented by Formula (1) described below:

f=tan(α/2)  (Formula 1)

An initial image is an image provided by determining a predeterminedarea T based on a preliminarily set initial setting and being producedbased on such a determined predetermined area T. An initial setting is,for example, (x, y, α)=(0, 0, 34) or the like.

At step S0706, the smartphone 2 causes a user to execute an operationfor switching to an image editing mode. Here, in a case where a userdoes not execute an operation for switching to an image editing mode,the smartphone 2 outputs, for example, an initial image.

At step S0707, the smartphone 2 executes a process for outputting anoutput image for editing an image.

FIG. 11 is a diagram that illustrates one example of an output image atan initial state for editing an image according to one embodiment of thepresent invention.

An output image is, for example, an output image 21 at an initial state.An output image has an editing image 31 at an initial state and achanging image 41 at an initial state.

An output image displays a button for a Graphical User Interface (GUI)for accepting an operation of a user. A GUI is, for example, a blurediting button 51, a cancellation editing button 52, or the like. Here,an output image may have another GUI.

An editing image 31 at an initial state is, for example, an initialimage produced at step S0705.

A changing image 41 at an initial state is, for example, an imageprovided by reducing an all celestial sphere panoramic image produced atstep S0703.

A user edits an image in an image editing mode, and hence, applies anoperation to an editing image or a changing image that is displayed inan output image.

At step S0708, the smartphone 2 executes a process for causing a user toinput an operation for editing an image.

At step S0709, the smartphone 2 acquires coordinates where a user inputsan operation for the input/output device 2H3. At step S0709, thesmartphone 2 executes a process for determining whether an operation isexecuted for an area of the editing image 31 at an initial state in FIG.11 or an operation is executed for an area of the changing image 41 atan initial state in FIG. 11, based on acquired coordinates.

Image editing is editing that is executed based on an operation of auser. Editing of an area to be output is editing for changing an area tobe output in an image based on a changing image or editing executed fora predetermined area based on an editing image.

Editing for changing an area to be output is executed in a case where anoperation is applied to an area of a changing image at step S0709.

Editing to be executed for a predetermined area based on an editingimage is executed in a case where an operation is applied to an area ofan editing image at step S0709.

In a case where a user operates a changing image (an area of a changingimage is determined at step S0709), the smartphone 2 goes to step S0710.In a case where a user operates an editing image (an area of an editingimage is determined at step S0709), the smartphone 2 goes to step S0712.

<Editing for Changing an Area to be Output>

FIG. 12A, FIG. 12B, and FIG. 12C are diagrams for illustrating oneexample of editing of an area to be output according to one embodimentof the present invention.

FIG. 12A is a diagram that illustrates one example of an output imageafter editing an area to be output according to one embodiment of thepresent invention.

An output image is, for example, an output image 22 after editing anarea to be output. The output image 22 after editing an area to beoutput has an editing image 32 after editing an area to be output and achanging image 42 after editing an area to be output.

The editing image 32 after editing an area to be output is an imageproduced by changing a predetermined area T as illustrated in FIG. 10A,FIG. 10B, FIG. 100, and FIG. 10D in the editing image 31 at an initialstate in FIG. 11.

The changing image 42 after editing an area to be output is an imageproduced by changing a predetermined area T illustrated in FIG. 10A,FIG. 10B, FIG. 100, and FIG. 10D in the changing image 41 at an initialstate in FIG. 11.

FIG. 12B is a diagram that illustrates one example of a predeterminedarea after editing an area to be output according to one embodiment ofthe present invention.

The output image 22 after editing an area to be output is provided at,for example, a viewpoint of a case where the virtual camera 3 at a stateof FIG. 10B is pan-rotated as illustrated in FIG. 12B.

FIG. 12C is a diagram that illustrates one example of an operation in acase of editing of an area to be output according to one embodiment ofthe present invention.

Editing of an area to be output is executed in such a manner that a useroperates a screen area where a changing image is output.

An operation to be input at step S0708 is, for example, an operation forchanging an area to be output with respect to left and right directionsof an image or the like.

In a case of FIG. 12A, FIG. 12B, and FIG. 12C, an operation that isinput by a user is such that a screen where the changing image 41 at aninitial state in FIG. 11 is traced with a finger in left and rightdirections of such a screen as illustrated in FIG. 12C, that is, aso-called “swipe operation”, or the like.

Herein, an input amount on a swipe operation is provided as (dx, dy).

A relation between a polar coordinate system (ϕ, θ) of an all celestialsphere in FIG. 8A, FIG. 8B, FIG. 8C, and FIG. 8D and an input amount(dx, dy) is represented by Formula (2) described below:

θ=k×dx

θ=k×dy  (Formula 2)

In Formula (2) described above, k is a predetermined constant forexecuting adjustment.

An output image is changed based on an input amount input for a swipeoperation, and hence, it is possible for a user to operate an image witha feeling that a sphere such as a terrestrial globe is rotated.

Here, for simplifying a process, what position of a screen a swipeoperation is input at may not be taken into consideration. That is,similar values may be input for an input amount (dx, dy) in Formula (2)even though a swipe operation is executed at any position of a screenwhere the changing image 41 at an initial state is output.

The changing image 42 after editing an area to be output executesperspective projection transformation of coordinates (Px, Py, Pz) of avertex P in three-dimensional space based on (ϕ, θ) calculated inaccordance with Formula (2).

In a case where a user executes a swipe operation with an input amount(dx2, dy2) in a case of FIG. 12A, a polar coordinate system (ϕ, θ) of anall celestial sphere is represented by Formula (3) described below:

ϕ=k×(dx+dx2)

θ=k×(dy+dy2)  (Formula 3)

As illustrated in (3) described above, a polar coordinate system (ϕ, θ)of an all celestial sphere is calculated based on a total value of inputamounts for respective swipe operations. Even in a case where aplurality of swipe operations are executed or the like, calculation of apolar coordinate system (ϕ, θ) of an all celestial sphere is executed,and thereby, it is possible to keep constant operability.

Here, editing of an area to be output is not limited to pan-rotation.For example, tilt-rotation of the virtual camera 3 in upper and lowerdirections of an image may be realized.

An operation that is input at step S0708 is, for example, an operationfor enlarging or reducing an area to be output or the like.

FIG. 13A and FIG. 13B are diagrams for illustrating one example ofenlargement or reduction of an area to be output according to oneembodiment of the present invention.

In a case where enlargement of an area to be output is executed, anoperation that is input by a user is such that two fingers are spread ona screen where the changing image 41 at an initial state in FIG. 11 isoutput, as illustrated in FIG. 13A, that is, a so-called “pinch-outoperation”, or the like.

In a case where reduction of an area to be output is executed, anoperation that is input by a user is such that two fingers are movedcloser to each other on a screen where the changing image 41 at aninitial state in FIG. 11 is output, as illustrated in FIG. 13B, that is,a so-called “pinch-in operation”, or the like.

Here, a pinch-out or pinch-in operation is sufficient as long as aposition where a finger of a user first contacts is provided in an areawith a changing image displayed thereon, and may be an operation thatsubsequently uses an area with an editing image displayed thereon.Furthermore, an operation may be executed by a so-called “stylus pen”that is a tool for operating a touch panel or the like.

In a case where an operation illustrated in FIG. 13A and FIG. 13B isinput, the smartphone 2 executes a so-called “zoom process”.

A zoom process is a process for producing an image with a predeterminedarea enlarged or reduced based on an operation that is input by a user.

In a case where an operation illustrated in FIG. 13A and FIG. 13B isinput, the smartphone 2 acquires an amount of change dz based on anoperation that is input by a user.

A zoom process is a process for executing calculation in accordance withFormula (4) described below:

α=α0+m×dz  (Formula 4)

based on an amount of change dz.

α indicated in Formula (4) described above is a view angle α of thevirtual camera 3 as illustrated in FIG. 10A, FIG. 10B, FIG. 10C, andFIG. 10D. m indicated in Formula (4) is a coefficient for adjusting anamount of zoom. α0 indicated in Formula (4) is a view angle α at aninitial state, that is, a view angle α in a case where an initial imageis produced at step S0705.

In a case where an operation illustrated in FIG. 13A and FIG. 13B isinput, the smartphone 2 determines a range of a predetermined area T inFIG. 10A, FIG. 10B, FIG. 100, and FIG. 10D by using a view angle αcalculated in accordance with Formula (4) for a projection matrix.

In a case where calculation is executed in accordance with Formula (4)and a user executes an operation for providing an amount of change dz2,the smartphone 2 executes calculation in accordance with Formula (5)described below:

α=α0+m×(dz+dz2)  (Formula 5)

As indicated in (5) described above, a view angle α is calculated basedon a total value of amounts of change due to operations as illustratedin FIG. 13A and FIG. 13B. Even in a case where a plurality of operationsas illustrated in FIG. 13A and FIG. 13B are executed or the like,calculation of a view angle α of a celestial sphere is executed, andthereby, it is possible to keep constant operability.

Here, a zoom process is not limited to a process in accordance withFormula (4) or Formula (5).

A zoom process may be realized by combining a view angle α of thevirtual camera 3 and a change in a position of a viewpoint.

FIG. 14 is a diagram for illustrating one example of another zoomprocess according to one embodiment of the present invention.

FIG. 14 is a model diagram for illustrating another zoom process. Asphere CS in FIG. 14 is similar to a sphere CS in FIG. 10A, FIG. 10B,FIG. 100, and FIG. 10D. In FIG. 14, a radius of a sphere CS is describedas “1”.

An origin in FIG. 14 is provided at an initial position of the virtualcamera 3. A position of the virtual camera 3 is changed on an opticalaxis, that is, a z-axis in FIG. 10A. It is possible to represent anamount of movement d of the virtual camera 3 by a distance from anorigin. For example, in a case where the virtual camera 3 is positionedat an origin, that is, a case of an initial state, an amount of movementd is “θ”.

A range of a predetermined area T in FIG. 10A, FIG. 10B, FIG. 100, andFIG. 10D is represented by an angle of view ω based on an amount ofmovement d and a view angle α of the virtual camera 3. An angle of viewω as illustrated in FIG. 14 is an angle of view in a case where thevirtual camera 3 is positioned at an origin, namely, a case of d=θ.

In a case where the virtual camera 3 is positioned at an origin, namely,a case of d=θ, an angle of view ω is identical to a view angle α. In acase where the virtual camera 3 is displaced from an origin, that is, acase where a value of d is increased, an angle of view ω and a viewangle α exhibit different ranges.

Another zoom process is a process for changing an angle of view ω.

FIG. 15 is a table for illustrating one example of another zoom processaccording to one embodiment of the present invention.

Illustrative table 4 illustrates an example of a case where an angle ofview ω is a range of 60° to 300°.

As illustrated in illustrative table 4, the smartphone 2 determineswhich of a view angle α and an amount of movement d of the virtualcamera 3 is preferentially changed based on a zoom specification valueZP.

“RANGE” is a range that is determined based on a zoom specificationvalue ZP.

“OUTPUT MAGNIFICATION” is an output magnification of an image calculatedbased on an image parameter determined by another zoom process.

“ZOOM SPECIFICATION VALUE ZP” is a value that corresponds to an angle ofview to be output. Another zoom process changes a process fordetermining an amount of movement d and a view angle α based on a zoomspecification value ZP. For a process to be executed in another zoomprocess, one of four methods is determined based on a zoom specificationvalue ZP as illustrated in illustrative table 4. A range of a zoomspecification value ZP is divided into four ranges that are a range ofA-B, a range of B-C, a range of C-D, and a range of D-E.

“ANGLE OF VIEW ω” is an angle of view ω that corresponds to an imageparameter determined by another zoom process.

“CHANGING PARAMETER” is a description that illustrates a parameter thatis changed by each of four methods based on a zoom specification valueZP. “REMARKS” are remarks for “CHANGING PARAMETER”.

“viewWH” in illustrative table 4 is a value that represents a width or aheight of an output area. In a case where an output area is laterallylong, “viewWH” is a value of a width. In a case where an output area islongitudinally long, “viewWH” is a value of a height. That is, “viewWH”is a value that represents a size of an output area in longitudinaldirection.

“imgWH” in illustrative table 4 is a value that represents a width or aheight of an output image. In a case where an output area is laterallylong, “imgWH” is a value of a width of an output image. In a case wherean output area is longitudinally long, “imgWH” is a value of a height ofan output image. That is, “imgWH” is a value that represents a size ofan output image in longitudinal direction.

“imageDeg” in illustrative table 4 is a value that represents an angleof a display range of an output image. In a case where a width of anoutput image is represented, “imageDeg” is 360°. In a case where aheight of an output image is represented, “imageDeg” is 180°.

FIG. 16A, FIG. 16B, FIG. 16C, FIG. 16D, and FIG. 16E are diagrams forillustrating one example of a “range” of another zoom process accordingto one embodiment of the present invention.

A case of a so-called “zoom-out” in FIG. 16A, FIG. 16B, FIG. 16C, FIG.16D, and FIG. 16E will be described as an example below. Here, a leftfigure in each figure of FIG. 16A, FIG. 16B, FIG. 16C, FIG. 16D, andFIG. 16E illustrates one example of an image to be output. A rightfigure in each figure of FIG. 16A, FIG. 16B, FIG. 16C, FIG. 16D, andFIG. 16E is a diagram that illustrates one example of a state of thevirtual camera 3 at a time of an output in a model diagram illustratedin FIG. 14.

FIG. 16A is one example of an output in a case where a zoomspecification value ZP is input in such a manner that a “RANGE” inillustrative table 4 in FIG. 15 is “A-B”. In a case of “A-B”, a viewangle α of the virtual camera 3 is fixed at, for example α=60°. In acase of “A-B”, an amount of movement d of the virtual camera 3 ischanged on a condition that a view angle α is fixed as illustrated inFIG. 16A. In a case where an amount of movement d of the virtual camera3 is increased on a condition that a view angle α is fixed, an angle ofview ω is increased. In a case of “A-B”, a view angle α is fixed and anamount of movement d of the virtual camera 3 is increased, so that it ispossible to realize a zoom-out process. Here, an amount of movement d ofthe virtual camera 3 in a case of “A-B” is from 0 to a radius of asphere CS. That is, a radius of a sphere CS is “1” in a case of FIG.16A, FIG. 16B, FIG. 16C, FIG. 16D, and FIG. 16E, and hence, an amount ofmovement d of the virtual camera 3 is a value within a range of 0-1. Anamount of movement d of the virtual camera 3 is a value that correspondsto a zoom specification value ZP.

FIG. 16B is one example of an output in a case where a zoomspecification value ZP is input in such a manner that “RANGE” inillustrative table 4 in FIG. 15 is “B-C”. “B-C” is a case where a zoomspecification value ZP is a value greater than that of “A-B”. In a caseof “B-C”, an amount of movement d of the virtual camera 3 is fixed at avalue for positioning the virtual camera 3 at a periphery of a sphereCS. That is, as illustrated in FIG. 16B, an amount of movement d of thevirtual camera 3 is fixed at “1” that is a radius of a sphere CS. In acase of “B-C”, a view angle α is changed on a condition that an amountof movement d of the virtual camera 3 is fixed. In a case where a viewangle α is increased on a condition that an amount of movement d of thevirtual camera 3 is fixed, an angle of view co is increased from FIG.16A to FIG. 16B. In a case of “B-C”, an amount of movement d of thevirtual camera 3 is fixed and a view angle α is increased, so that it ispossible to realize a zoom-out process. In a case of “B-C”, a view angleα is calculated as ω/2. In a case of “B-C”, a range of a view angle α isfrom 60° that is a value fixed in a case of “A-B” to 120°.

In a case of “A-B” or “B-C”, an angle of view ω is identical to a zoomspecification value ZP. In a case of “A-B” or “B-C”, a value of an angleof view ω is increased.

FIG. 16C is one example of an output in a case where a zoomspecification value ZP is input in such a manner that “RANGE” inillustrative table 4 in FIG. 15 is “C-D”. “C-D” is a case where a zoomspecification value ZP is a value greater than that of “B-C”. In a caseof “C-D”, a view angle α is fixed at, for example, α=120°. In a case of“C-D”, an amount of movement d of the virtual camera 3 is changed on acondition that a view angle α is fixed as illustrated in FIG. 16C. In acase where an amount of movement d of the virtual camera 3 is increasedon a condition that a view angle α is fixed, an angle of view ω isincreased. An amount of movement d of the virtual camera 3 is calculatedin accordance with a formula based on a zoom specification value ZPillustrated in illustrative table 4 I FIG. 15. In a case of “C-D”, anamount of movement d of the virtual camera 3 is changed to a maximumdisplay distance dmax1.

A maximum display distance dmax1 is a distance where a sphere CS isdisplayed so as to be maximum in an output area of the smartphone 2. Anoutput area is, for example, a size of a screen where the smartphone 2outputs an image or the like, or the like. A maximum display distancedmax1 is, for example, a case of FIG. 16D. A maximum display distancedmax1 is calculated in accordance with Formula (6) described below:

$\begin{matrix}{{d\; \max \; 1} = \frac{1}{\sin \left\{ {a\; {\tan \left\lbrack {{\tan \left( \frac{\omega}{2} \right)}*\frac{\left( {{viewW}^{2} + {viewH}^{2}} \right)^{\frac{1}{2}}}{viewW}} \right\rbrack}} \right\}}} & \left( {{Formula}\mspace{14mu} 6} \right)\end{matrix}$

“viewW” in Formula (6) described above is a value that represents awidth of an output area of the smartphone 2. “viewH” in Formula (6)described above is a value that represents a height of an output area ofthe smartphone 2. A similar matter will be described below.

A maximum display distance dmax1 is calculated based on values of“viewW” and “viewH” that are output areas of the smartphone 2.

FIG. 16D is one example of an output in a case where a zoomspecification value ZP is input in such a manner that “RANGE” inillustrative table 4 in FIG. 15 is “D-E”. “D-E” is a case where a zoomspecification value ZP is a value greater than that of “C-D”. In a caseof “D-E”, a view angle α is fixed at, for example, α=120°. In a case of“D-E”, an amount of movement d of the virtual camera 3 is changed on acondition that a view angle α is fixed as illustrated in FIG. 16D. Anamount of movement d of the virtual camera 3 is changed to a limitdisplay distance dmax2. A limit display distance dmax2 is a distancewhere a sphere CS is displayed so as to be inscribed in an output areaof the smartphone 2. A limit display distance dmax2 is calculated inFormula (7) described below:

$\begin{matrix}{{d\; \max \; 2} = \frac{1}{\sin \left\{ {a\; {\tan \left\lbrack {{\tan \left( \frac{\omega}{2} \right)}*\frac{viewH}{viewW}} \right\rbrack}} \right\}}} & \left( {{Formula}\mspace{14mu} 7} \right)\end{matrix}$

A limit display distance dmax2 is, for example, a case of FIG. 16E.

A limit display distance dmax2 is calculated based on values of “viewW”and “viewH” that are output areas of the smartphone 2. A limit displaydistance dmax2 represents a maximum range that is able to be output bythe smartphone 2, that is, a limit value of an amount of movement d ofthe virtual camera 3. An embodiment may be limited in such a manner thata zoom specification value ZP is included in a range illustrated inillustrative table 4 in FIG. 15, that is, a value of an amount ofmovement d of the virtual camera 3 is less than or equal to a limitdisplay distance dmax2. Due to such limitation, the smartphone 2 isprovided on a condition that an output image is fitted to a screen thatis an output area or a condition that an image with a predeterminedoutput magnification is output to a user, so that it is possible torealize zoom-out.

Due to a process for “D-E”, it is possible for the smartphone 2 to causea user to recognize that an output image is all celestial spherepanorama.

Here, in a case of “C-D” or “D-E”, an angle of view ω is not identicalto a zoom specification value ZP. Furthermore, as illustrated inillustrative table 4 in FIG. 15 and FIG. 16A, FIG. 16B, FIG. 16C, FIG.16D, and FIG. 16E, an angle of view ω is continuous in each range butsuch an angle of view ω is not uniformly increased by zoom-out toward awide-angle side. That is, in a case of “C-D” where an amount of movementd of the virtual camera 3 is changed, an angle of view ω is increasedwith such an amount of movement d of the virtual camera 3. In a case of“D-E” where an amount of movement d of the virtual camera 3 is changed,an angle of view ω is decreased with such an amount of movement d of thevirtual camera 3. A decrease in an amount of movement d of the virtualcamera 3 in “D-E” is caused by reflecting an outer area of a sphere CS.In a case where a wide field of view greater than or equal to 240° isspecified by a zoom specification value ZP, the smartphone 2 changes anamount of movement d of the virtual camera 3, and thereby, it ispossible to output an image with a less feeling of strangeness to a userand change an angle of view w.

In a case where a zoom specification value ZP is changed toward awide-angle direction, an angle of view ω is frequently increased. In acase where an angle of view ω is increased, the smartphone 2 fixes aview angle α of the virtual camera 3 and increases an amount of movementd of the virtual camera 3. The smartphone 2 fixes a view angle α of thevirtual camera 3, and thereby, it is possible to reduce an increase insuch a view angle α of the virtual camera 3. The smartphone 2 reduces anincrease in a view angle α of the virtual camera 3, and thereby, it ispossible to output an image with less distortion to a user. In a casewhere a view angle α of the virtual camera 3 is fixed, the smartphone 2increases an amount of movement d of the virtual camera 3, that is,moves the virtual camera 3 to be distant, and thereby, it is possible toprovide a user with an open-feeling of a wide angle display.Furthermore, movement for moving the virtual camera 3 to be distant issimilar to movement at a time when a human being confirms a wide range,and hence, it is possible for the smartphone 2 to realize zoom-out witha less feeling of strangeness due to movement for moving the virtualcamera to be distant.

In a case of “D-E”, an angle of view m is decreased with changing a zoomspecification value ZP toward a wide-angle direction. In a case of“D-E”, the smartphone 2 decreases an angle of view w, and thereby, it ispossible to provide a user with a feeling of being distant from a sphereCS. The smartphone 2 provides a user with a feeling of being distantfrom a sphere CS, and thereby, it is possible to output an image with aless feeling of strangeness to a user.

Hence, it is possible for the smartphone 2 to output an image with aless feeling of strangeness to a user, due to another zoom processillustrated in illustrative table 4 in FIG. 15.

Here, an embodiment is not limited to a case where only an amount ofmovement d or a view angle α of the virtual camera 3 illustrated inillustrative table 4 in FIG. 15 is changed. It is sufficient for anembodiment to be a mode for preferentially changing an amount ofmovement d or a view angle α of the virtual camera 3 on a conditionillustrated in illustrative table 4 in FIG. 15, and a fixed value may bechanged to a sufficiently small value, for example, for adjustment.

Furthermore, an embodiment is not limited to zoom-out. An embodiment mayrealize, for example, zoom-in.

Here, a case where an area to be output is edited is not limited to acase where an operation is executed for a changing image. The smartphone2 may edit an area to be output, for example, in a case where anoperation is executed for an editing image.

<Editing to be Executed for a Predetermined Area Based on an EditingImage>

Editing to be executed for a predetermined area based on an editingimage is blur editing that blurs a predetermined pixel. Herein, foranother editing, it is possible to provide, erasing of a specified rangeof an image, changing of a color tone or a color depth of an image orthe like, a color change of a specified range of an image, or the like.

A case where a user executes blur editing for the output image 22 afterediting of an area to be output in FIG. 12A, FIG. 12B, and FIG. 12C willbe described as an example below.

In a case where a user executes an operation that pushes a blur editingbutton 51, the smartphone 2 causes a user to input a so-called “tapoperation” for an area where an editing image 32 for the output image 22after editing of an area to be output in FIG. 12A, FIG. 12B, and FIG.12C is displayed.

The smartphone 2 executes a process for blurring a predetermined rangecentered at a point tapped by a user.

FIG. 17A and FIG. 17B are diagrams for illustrating one example ofediting to be executed for a predetermined area based on an editingimage according to one embodiment of the present invention.

FIG. 17A is a diagram for illustrating one example of blur editingaccording to one embodiment of the present invention. FIG. 17A is adiagram that illustrates an output image 23 after blur editing. Theoutput image 23 after blur editing has an editing image 33 after blurediting and a changing image 43 after blur editing.

The editing image 33 after blur editing is produced by applying blurediting to an output image after editing of an area to be output in FIG.12A, FIG. 12B, and FIG. 12C. Blur editing is realized by, for example, aGauss function, an average of peripheral pixels, a low-pass filter, orthe like. Blur editing is illustrated like, for example, a blur editingarea 5.

Blur editing is applied to a changing image. The smartphone 2 executedcalculation of a point (Px, Py, Pz) in a three-dimensional space fromcoordinates of a point tapped by a user. The smartphone 2 calculates(Px, Py, Pz) from two-dimensional coordinates through inversetransformation of perspective projection transformation that uses a viewfrustum. There is no information of depth in two-dimensionalcoordinates, and hence, (Px, Py, Pz) are calculated by using a point ona sphere and simultaneous equations. A sign of Pz in a projectioncoordinate system is constant, and hence, it is possible for thesmartphone 2 to calculate simultaneous equations. Coordinates of an allcelestial sphere panoramic image correspond to (Px, Py, Pz), and hence,it is possible for the smartphone 2 to calculate coordinates on an allcelestial sphere panoramic image from calculated (Px, Py, Pz).Therefore, a changing image 43 after blur editing is provided on acondition that blur editing is reflected as illustrated in FIG. 17A.

FIG. 17B is a diagram for illustrating one example of editing thatcancels blurring according to one embodiment of the present invention.

Editing that is applied to a predetermined area based on an editingimage is editing that cancels blur editing for a blur editing area 5blurred by such blur editing.

In a case where a user executes an operation that pushes thecancellation editing button 52, the smartphone 2 outputs an output image24 for cancellation editing that displays a filling area 6 on the blurediting area 5 with applied blur editing. As illustrated in FIG. 17B,the output image 24 for cancellation editing is an image that displaysthe filling area 6 on the blur editing area 5 in the editing image 33after blur editing in FIG. 17A. A user executes a tap operation for adisplayed filling area 6, that is, an area with applied blurring. Thesmartphone 2 executes a process for cancelling blur editing in apredetermined range centered at a point tapped by a user. That is, inediting for cancelling blur editing, the smartphone 2 provides apredetermined range centered at a point tapped by a user in the editingimage 33 after blur editing on a state of the output image 22 afterediting of an area to be output in FIG. 12A, FIG. 12B, and FIG. 12C.

Once a taken image of a face of a person or a photography-prohibitedbuilding is released or shared on the internet, trouble may be caused.In particular, in a case where a panoramic image with a broad range istaken, an image of many objects in a broad range may frequently betaken. Therefore, it is possible for a user to reduce trouble due to aprocess for blurring an object that is possibly problematic at a time ofrelease or sharing. It is possible for the smartphone 2 to facilitate anoperation for blurring a face of a person taken in an image due toediting to be applied to a predetermined area based on an editing image.Hence, it is possible for the smartphone 2 to cause a user to readilyexecute an image operation due to editing to be applied to apredetermined area based on an editing image.

Here, in a case where editing of an area to be output is executed, thesmartphone 2 may change a range of editing applied to a predeterminedarea based on an editing image or the like in accordance with amagnification.

At step S0710, the smartphone 2 calculates amounts of movement ofcoordinates to be output. That is, at step S0710, the smartphone 2calculates a position of a predetermined area T in FIG. 10A, FIG. 10B,FIG. 100, and FIG. 10D that corresponds to a swipe operation of a userbased on, for example, Formula (2) described above.

At step S0711, the smartphone 2 updates a position of a predeterminedarea T in FIG. 10A, FIG. 10B, FIG. 100, and FIG. 10D at a positioncalculated at step S0710.

At step S0712, the smartphone 2 calculates coordinates of a point thatis an editing object. That is, at step S0712, the smartphone 2calculates coordinates that correspond to a tap operation of a user andexecutes calculation for projection onto three-dimensional coordinates.

At step S0713, the smartphone 2 calculates a predetermined area that isedited centered at coordinates calculated at step S0712 and based on anediting image. That is, at step S0713, the smartphone 2 calculates apixel that is a point specified by a tap operation of a user or aperiphery of such a point and is an object for blur editing or the like.

At step S0714, the smartphone 2 produces an editing image. In a casewhere a user executes an operation for a changing image at step S0714,the smartphone 2 produces a changing image based on a predetermined areaT updated at step S0711. In a case where a user executes an operationfor an editing image at step S0714, the smartphone 2 produces an editingimage wherein a blurring process is reflected on a pixel calculated atstep S0713.

At step S0715, the smartphone 2 produces a changing image. In a casewhere a user executes an operation for a changing image at step S0715,the smartphone 2 produces a changing image based on a predetermined areaT updated at step S0711. In a case where a user executes an operationfor an editing image at step S0715, the smartphone 2 produces anchanging image that indicates a location that is a blurring object atstep S713.

The smartphone 2 repeats processes of step S0708 through step S0715.

<A Process on a Smartphone>

FIG. 18 is a flowchart that illustrates one example of an entire processon a smartphone according to one embodiment of the present invention.

At step S1801, the smartphone 2 executes a process for acquiring animage from the image taking device 1 in FIG. 1 or the like. A process atstep S1801 corresponds to a process at step S0702 in FIG. 7.

At step S1802, the smartphone 2 executes a process for producing apanoramic image. A process at step S1802 is executed based on an imageacquired at step S1801. A process at step S1802 corresponds to a processat step S0703 in FIG. 7.

At step S1803, the smartphone 2 executes a process for causing a user toselect an image to be output. A process at step S1803 corresponds to aprocess at step S0704 in FIG. 7. Specifically, a process for causing auser to select an image to be output is a process for outputting athumbnail image or providing a UI for causing a user to execute anoperation for a thumbnail image, or the like.

At step S1804, the smartphone 2 executes a process for producing aninitial image. A process at step S1804 corresponds to a process at stepS0705 in FIG. 7. At step S1804, the smartphone 2 produces and outputs animage selected by a user at step S1803 as an initial image.

At step S1805, the smartphone 2 executes determination as to whether ornot switching to a mode for editing an image is executed. A process atstep S1805 executes determination based on whether or not an operationof a user at step S0706 in FIG. 7 is provided. In a case wheredetermination is provided at step S1805 in such a manner that switchingto a mode for editing an image is provided (YES at step S1805), thesmartphone 2 goes to step S1806. In a case where determination isprovided at step S1805 in such a manner that switching to a mode forediting an image is not provided (NO at step S1805), the smartphone 2returns to step S1804.

A case where determination is provided at step S1805 in such a mannerthat switching to a mode for editing an image is provided is a casewhere an input to start editing of an image is provided by a user. Acase where determination is provided at step S1805 in such a manner thatswitching to a mode for editing an image is not provided is a case wherea user does not execute an operation. Therefore, in a case where a userdoes not execute an operation, the smartphone 2 continues to output aninitial image and waits for an input of an user to start editing of animage.

At step S1806, the smartphone 2 executes a process for outputting anoutput image for editing an image. A process at step S1806 correspondsto a process at step S0707 in FIG. 7. Furthermore, the smartphone 2outputs an output image and thereby accepts an operation of a user atstep S0708 in FIG. 7.

At step S1807, the smartphone 2 executes determination as to whether anoperation of a user is executed for an editing image or a changingimage. A process at step S1807 corresponds to a process at step S0709 inFIG. 7. The smartphone 2 executes determination as to whether anoperation of a user at step S0708 in FIG. 7 is executed for an editingimage or a changing image.

In a case where determination is provided in such a manner that anoperation of a user is executed for a changing image (a changing imageat step S1807), the smartphone 2 goes to step S1808. In a case wheredetermination is provided in such a manner that an operation of a useris executed for an editing image (an editing image at step S1807), thesmartphone 2 goes to step S1810.

At step S1808, the smartphone 2 executes a process for calculating anamount of movement of a predetermined area due to an operation. Aprocess at step S1808 corresponds to a process at step S0710 in FIG. 7.At step S1808, the smartphone 2 calculates an amount of movement formoving a predetermined area based on a swipe operation that is executedby a user and changes such a predetermined area.

At step S1809, the smartphone 2 executes a process for updating apredetermined area. A process at step S1809 corresponds to a process atstep S0711 in FIG. 7. At step S1809, the smartphone 2 moves apredetermined area T in FIG. 10A, FIG. 10B, FIG. 10C, and FIG. 10D to aposition that corresponds to an amount of movement calculated at stepS1808, and updates such a predetermined area T from a position of aninitial image to a position that corresponds to a swipe operation of auser.

At step S1810, the smartphone 2 executes a process for calculating, andthree-dimensionally projecting, coordinates that are objects for anoperation. A process at step S1810 corresponds to a process at stepS0712 in FIG. 7. At step S1810, the smartphone 2 calculates coordinateson an all celestial sphere image that corresponds to a point specifiedby a tap operation of a user.

At step S1811, the smartphone 2 executes a process for calculating apixel that is an object for blurring. For example, the smartphone 2 hasan editing state table that causes flag data as to whether or not anobject for blurring is provided, to correspond to each pixel. An editingstate table represents whether or not each pixel is output in a blurstate. The smartphone 2 refers to an editing state table, determineswhether or not each pixel in an output image is output in a blur state,and outputs an image. That is, a process at step S1811 is a process forupdating an editing state table. In a case where an operation for eitherblurring as illustrated in FIG. 17A or cancellation as illustrated inFIG. 17B is provided in a tap operation of a user, the smartphone 2updates an editing state table based on such an operation.

At step S1812, the smartphone 2 executes a process for producing anediting image. A process at step S1812 corresponds to a process at stepS0714 in FIG. 7.

At step S1813, the smartphone 2 executes a process for producing achanging image. A process at step S1813 corresponds to a process at stepS0715 in FIG. 7.

Due to processes at step S1812 and step S1813, the smartphone 2 producesan output image and executes an output to a user.

The smartphone 2 returns to step S1807 and repeats previouslyillustrated processes.

In a case where an object for blurring is provided based on an editingstate table at processes at step S1812 and step S1813, the smartphone 2executes, for example, a blurring process as illustrated in FIG. 17A andan output.

An image that is output to a user by the smartphone 2 is output at 30 ormore frames per 1 second in such a manner that such a user feels smoothreproduction of an animation. It is desirable for the smartphone 2 toexecute an output at 60 or more frames per 1 second in such a mannerthat a user feels particularly smooth reproduction. Here, a frame rateof an output may be such that 60 frames per 1 second is changed to, forexample, 59.94 frames per 1 second.

Here, processes at step S1812 and step S1813 are not limited toprocesses for causing the smartphone 2 to execute a blurring process andan output.

For example, the smartphone 2 has an image provided by preliminarilyapplying a blurring process to all of pixels of an image to be outputand an image provided by applying no blurring process. The smartphone 2outputs each pixel by simultaneously selecting an image provided byexecuting a blurring process based on an editing state table or an imageprovided by executing no blurring process. It is possible for thesmartphone 2 to reduce an amount of calculation for outputting an imageby preliminarily executing a blurring process. That is, it is possiblefor the smartphone 2 to realize a high-speed image output such as 60frames per 1 second by executing selection and a simultaneous output ofeach pixel.

Furthermore, for example, in a case where each pixel is selected and asimultaneously output, the smartphone 2 may store an output image. In acase where a user does not execute an editing operation, the smartphone2 outputs a stored image. Due to storage, a process for selecting andproducing each pixel of an image to be output is not required, andhence, it is possible for the smartphone 2 to reduce an amount ofcalculation. Therefore, the smartphone 2 stores an output image, andthereby, it is possible to realize a high-speed image output such as 60frames per 1 second.

Here, an output image is not limited to an image illustrated in FIG. 11or the like. For example, a shape, a position, a size, or a range of anediting image or a changing image may be changed.

FIG. 19A and FIG. 19B are diagrams for illustrating one example ofchanging of an output such as a position, a direction, or the like, of achanging image according to one embodiment of the present invention.

An information processing device that is one example of a device fordisplaying an output image is, for example, the smartphone 2. Thesmartphone 2 will be described as an example below.

FIG. 19A is a diagram that illustrates one example of changing of anattitude of the smartphone 2 according to one embodiment of the presentinvention.

For example, a changing image is output to a position 7 before changingas illustrated in FIG. 19A. A case of FIG. 19A will be described as anexample below.

For example, an attitude of the smartphone 2 is changed by a user in adirection of rotation as illustrated in FIG. 19A. An attitude of thesmartphone 2 is detected by the state sensor 2H4 in FIG. 6. Thesmartphone 2 rotates and outputs an output image based on an attitude ofthe smartphone 2 that is a result of detection. The smartphone 2 maychange a position or a direction of an area for outputting a changingimage based on a result of detection.

FIG. 19B is a diagram that illustrates one example that changes aposition or a direction of an area where a changing image is displayedbased on a result of detection, according to one embodiment of thepresent invention.

In a case where a user rotates the smartphone 2 as illustrated in FIG.19A, the smartphone 2 changes a position of an area for outputting achanging image from a position illustrated as the position 7 beforechanging in FIG. 19A to a first changing position 71 or a secondchanging position 72.

Here, a changing image may be output on a condition that a direction foran output is changed based on a result of detection as illustrated inFIG. 19B, that is, rotated from a state of FIG. 19A to that asillustrated in FIG. 19B.

The smartphone 2 changes a position or a direction of an area foroutputting a changing image based on a result of detection. That is, itis possible for the smartphone 2 to output an image at a position or ina direction for facilitating an operation of a user even if an attitudeof the smartphone 2 is changed, in order to output such an image inaccordance with such an attitude.

Furthermore, for changing of a position or a direction of a changingimage, the smartphone 2 may display such a changing image so as tobounce during such changing.

<A Functional Configuration>

FIG. 20 is a block diagram that illustrates one example of a functionalconfiguration of an image taking system according to one embodiment ofthe present invention.

The image taking system 10 has the image taking device 1 and thesmartphone 2. The image taking system 10 has a first image taking part1F1, a second image taking part 1F2, and an all celestial sphere imageproduction part 1F3. The image taking system 10 has an image acquisitionpart 2F1, a production part 2F2, an input/output part 2F3, a detectionpart 2F4, a storage part 2F5, and a control part 2F6.

The first image talking part 1F1 and the second image taking part 1F2take and produce images that are materials of an all celestial sphereimage. The first image taking part 1F1 is realized by, for example, thefront side image taking element 1H1 in FIG. 2A, FIG. 2B, and FIG. 2C orthe like. The second image taking part 1F2 is realized by, for example,the back side image taking element 1H2 in FIG. 2A, FIG. 2B, and FIG. 2Cor the like. An image that is a material of an all celestial sphereimage is, for example, a hemispherical image as illustrated in FIG. 4Aor FIG. 4B.

The all celestial sphere image production part 1F3 produces an imagethat is output to the smartphone 2, such as an all celestial sphereimage. The all celestial sphere image production part 1F3 is realizedby, for example, the image processing unit 1H7 in FIG. 5 or the like.The all celestial sphere image production part 1F3 produces an allcelestial sphere image from hemispherical images that are taken by thefirst image taking part 1F1 and the second image taking part 1F2.

The image acquisition part 2F1 acquires image data such as an allcelestial sphere image from the image taking device 1. The imageacquisition part 2F1 is realized by, for example, the network I/F 2H6 inFIG. 6 or the like. The image acquisition part 2F1 executes a processfor causing the smartphone 2 to acquire image data such as an allcelestial sphere image.

The production part 2F2 executes a process for producing each kind ofimage and each kind of calculation necessary for production of an image.The production part 2F2 has a changing image production part 2F21 and anediting image production part 2F22. The production part 2F2 is realizedby the CPU 2H5 in FIG. 6 or the like.

The changing image production part 2F21 executes a process for executingproduction of a changing image. The changing image production part 2F21acquires, for example, image data and an editing state table from thestorage part 2F5. The changing image production part 2F21 produces achanging image based on an acquired editing state table and image data.

The editing image production part 2F22 executes a process for executingproduction of an editing image. The editing image production part 2F22acquires, for example, image data and an editing state table from thestorage part 2F5. The editing image production part 2F22 produces anediting image based on an acquired editing state table and image data.

The production part 2F2 calculates, and stores as an editing statetable, coordinates associated with an operation in a case where a userexecutes a tap or swipe operation. Furthermore, an image produced by theproduction part 2F2 may be stored in the storage part 2F5 and takenaccording to a process.

The production part 2F2 may produce each kind of image based on a resultof detection that is acquired from the detection part 2F4.

The input/output part 2F3 executes a process for inputting an operationof a user. The input/output part 2F3 causes a user to execute a processfor outputting an image produced by the production part 2F2. Theinput/output part 2F3 is realized by, for example, the input/outputdevice 2H3 in FIG. 6 or the like.

The detection part 2F4 executes a process for detecting an attitude ofthe smartphone 2. The detection part 2F4 is realized by, for example,the state sensor 2H4 in FIG. 6 or the like.

The storage part 2F5 stores each kind of information acquired orproduced by the smartphone 2. The storage part 2F5 has, for example, anediting state table storage part 2F51 and an image storage part 2F52.The storage part 2F5 is realized by, for example, the auxiliary storagedevice 2H1 or the main storage device 2H2 in FIG. 6 or the like.

The editing state table storage part 2F51 stores data of a table thatrepresents a pixel where a blurring process is executed.

The image storage part 2F52 stores an all celestial sphere imageacquired by the image acquisition part 2F1, an output image produced bythe production part 2F2, and the like.

The control part 2F6 controls each kind of a component that is providedin the smartphone 2. The control part 2F6 controls each kind ofcomponent, and thereby, realizes each kind of process, a process forassisting each kind of process, and the like. The control part 2F6 isrealized by, for example, the CPU 2H5 in FIG. 6 or the like.

Here, an entire process is not limited to a case as illustrated in FIG.7. For example, a part or an entirety of each process may be processedby a device other than a device as illustrated in FIG. 7.

The smartphone 2 produces an editing image and a changing image based onan all celestial sphere image acquired from the image taking device 1 orthe like. An editing image is an image for outputting a predeterminedarea that is determined by a predetermined area T and causes a user toexecute an editing operation such as blurring or cancellation ofblurring. A changing image is an image for causing a user to execute anoperation for changing a position, a size, or a range of a predeterminedarea T, or the like. The smartphone 2 outputs an output image that hasat least an editing image and a changing image. An output image has anediting image and a changing image, and thereby, it is possible for thesmartphone 2 to cause a user to execute editing such as blurring andsimultaneously change an area output in such an editing image by such achanging image. Therefore, in a case where a user executes a blurringoperation for an all celestial sphere image or the like, it is possiblefor the smartphone 2 to output an image for facilitating an operation.Hence, the smartphone 2 outputs an output image that has an editingimage and a changing image, and thereby, it is possible for a user toreadily execute an operation of an image.

Here, the smartphone 2 may be realized by a computer-executable programdescribed in a legacy programing language such as Assembler, C, C++, C#,or Java (“Java” is a registered trademark), an object-orientedprogramming language, or the like. It is possible for a program to bestored in and distributed by a recording medium such as a ROM or anElectrically Erasable Programmable ROM (EEPROM). It is possible for aprogram to be stored in and distributed by a recording medium such as anErasable Programmable ROM (EPROM). It is possible for a program to bestored in and distributed by a recording medium such as a flash memory,a flexible disk, a CD-ROM, a CD-RW, a DVD-ROM, a DVD-RAM, a DVD-RW, orthe like. It is possible for a program to be stored in a device-readablerecording medium such as a Blu-Ray disk (“Blu-Ray disk” is a registeredtrademark), SD (“SD” is a registered trademark) card, or an MO ordistributed through a telecommunication line.

Here, an image in an embodiment is not limited to a still image. Forexample, an image may be an animation.

Furthermore, a part or an entirety of each process in an embodiment maybe realized by, for example, a programmable device (PD) such as a fieldprogrammable gate array (FPGA). Moreover, a part or an entirety of eachprocess in an embodiment may be realized by an Application SpecificIntegrated Circuit (ASIC).

Although preferable practical examples of the present invention havebeen described in detail above, the present invention is not limited tosuch particular embodiments and a variety of alterations andmodifications are possible within a scope of an essence of the presentinvention as recited in what is claimed.

[Appendix]

<An illustrative embodiment(s) of an information processing method, aninformation processing device, and a program>

At least one illustrative embodiment of the present invention may relateto an information processing method, an information processing device,and a program.

At least one illustrative embodiment of the present invention may aim atfacilitating execution of an image operation for a user.

According to at least one illustrative embodiment of the presentinvention, there may be provided an information processing method thatcauses a computer to process an image, characterized by causing thecomputer to execute an acquisition step for acquiring the image, aproduction step for producing an editing image for editing apredetermined area of the image and a changing image for changing thepredetermined area to be output, and an output step for outputting anoutput image that has at least the editing image and the changing image.

Illustrative embodiment (1) is an information processing method forcausing a computer to process an image, wherein the image processingmethod causes the computer to execute an acquisition step for acquiringthe image, a production step for producing an editing image for editinga predetermined area of the image and a changing image for changing thepredetermined area to be output, and an output step for outputting anoutput image that has at least the editing image and the changing image.

Illustrative embodiment (2) is the information processing method asdescribed in illustrative embodiment (1), wherein an editing area inputstep for acquiring an editing area that is a target area of the editingby using the editing image and an editing step for editing the editingarea are executed.

Illustrative embodiment (3) is the image processing method as describedin illustrative embodiment (2), wherein the editing step is a step forblurring the editing area.

Illustrative embodiment (4) is the information processing method asdescribed in illustrative embodiment (3), wherein an acquisition imagethat has just been acquired in the acquisition step and a blurred imageproduced by a blurring process are produced and the output image isoutput by selecting a pixel of the blurred image for the editing areaand a pixel of the acquisition image for that other than the editingarea.

Illustrative embodiment (5) is the information processing method asdescribed in any one of illustrative embodiments (2) to (4), wherein aspecified area input step for acquiring a specifying area for specifyingan area of a part or an entirety of an image output with the editingimage and a cancellation step for canceling the editing process executedfor the specifying area are executed.

Illustrative embodiment (6) is the information processing method asdescribed in any one of illustrative embodiments (1) to (5), wherein anoperation input step for acquiring an operation for changing, enlarging,or reducing the predetermined area that is output with the editing imageby using the changing image is executed.

Illustrative embodiment (7) is the information processing method asdescribed in illustrative embodiment (6), wherein a determination stepfor determining a view point position and a view angle is executed basedon the operation and the determination changes one of the view pointposition and the view angle based on an area indicated by the operation.

Illustrative embodiment (8) is the information processing method asdescribed in any one of illustrative embodiments (1) to (7), wherein adetection step for detecting an attitude of a device that displays theoutput image and a changing step for changing a position or direction ofthe changing image based on a result of detection by the detection stepare executed.

Illustrative embodiment (9) is an information processing device thatprocesses an image, wherein the image processing device has anacquisition means for acquiring the image, a production means forproducing an editing image for editing a predetermined area of the imageand a changing image for changing the predetermined area to be output,and an output means for outputting an output image that has at least theediting image and the changing image.

Illustrative embodiment (10) is a program for causing a computer toprocess an image, wherein the program causes the computer to execute anacquisition step for acquiring the image, a production step forproducing an editing image for editing a predetermined area of the imageand a changing image for changing the predetermined area to be output,and an output step for outputting an output image that has at least theediting image and the changing image.

According to at least an illustrative embodiment of the presentinvention, it may be possible to facilitate execution of an imageoperation for a user.

Although the illustrative embodiment(s) and specific example(s) of thepresent invention have been described with reference to the accompanyingdrawings, the present invention is not limited to any of theillustrative embodiment(s) and specific example(s) and the illustrativeembodiment(s) and specific example(s) may be altered, modified, orcombined without departing from the scope of the present invention.

All examples and conditional language recited herein are intended forpedagogical purposes to aid the reader in understanding the principlesof the invention and the concepts contributed by the inventor tofurthering the art, and are to be construed as being without limitationto such specifically recited examples and conditions, nor does theorganization of such examples in the specification relate to a showingof the superiority or inferiority of the invention. Although aninformation processing method has been described in detail, it should beunderstood that various changes, substitutions, and alterations could bemade thereto without departing from the spirit and scope of theinvention.

What is claimed is:
 1. An information processing method for causing acomputer to process an image, wherein the image processing method causesthe computer to execute: an first display step for displaying a firstimage including at least a portion of the image an all celestial sphereimage; a second display step for displaying a second image included inthe first image; an editing step for editing the second image; and aninput operation receiving step for receiving an input operation from auser to the first image or the second image.
 2. The informationprocessing method according to claim 1, wherein the image processingmethod causes the computer to further execute: a third display step fordisplaying images of the first image and the second image, the imagesbeing changed by the input operation received by the input operationreceiving step.
 3. A non-transitory computer-readable storage mediumstoring a program for causing a computer to execute an informationprocessing method including: an first display step for displaying afirst image including at least a portion of an all celestial sphereimage; a second display step for displaying a second image included inthe first image; an editing step for editing the second image; and aninput operation receiving step for receiving an input operation from auser to the first image or the second image.
 4. An informationprocessing method for causing a computer to process an image, whereinthe image processing method causes the computer to execute: a displaystep for displaying a first image including at least a portion of an allcelestial sphere image in a first area and a second image included inthe first image in a second area; an editing step for editing the secondimage; and an input operation receiving step for receiving an inputoperation from a user to the first image or the second image.